The butterfly effect, shown comically in figure 1, is the idea that a very small change in one place on Earth can cause a very big change somewhere else. In this case, a butterfly flaps its wings and causes a tornado. This metaphor illustrates the mathematical concept of chaos, in which the Earth’s atmosphere is a chaotic system. While a single butterfly probably isn’t literally responsible for a tornado, mathematical chaos is very real and important. So this week, I’m going to try giving you some intuition for the butterfly effect using one extreme example from physics.

Heat

Suppose we take a flat, rectangular piece of metal and heat it up at four specific spots. Figure 2 shows what will happen to the metal: The four hot spots (shown in red at the start) will cool off as the heat spreads out, diffusing across the metal until the whole piece reaches the same temperature.

If we isolated the piece of metal beforehand, no heat can “escape” it, so it will never cool back down to its original temperature. The total amount of energy in the system will stay the same. The only thing that changes is how the heat is distributed over the metal’s surface. This “flow” of heat is described by the heat equation. Given any distribution of temperature across the metal, we can use the heat equation to know how hot each area of the metal will be at any point in the future.

But what if, instead of making a prediction about the future, we want to make a postdiction? What if we want to know the temperature of the metal at some point in the past?

Heat Flow Backwards?

Of course, we know the temperature change originated at the four spots we heated up, but let’s pretend we don’t. Suppose that we only saw our metal piece after its whole surface had reached the same temperature. Furthermore, suppose that we’re just a little uncertain about the temperature of the metal now. Maybe there are a few spots that are slightly hotter or colder than average—say, from us touching it, or from sunlight. Probably the best way to figure out what the metal looked like in the past is to take our best guess as to the temperature now, feed that number into the heat equation, and run it in reverse, right?

I did exactly that and figure 3 shows the result.

That doesn’t look anything like the four dots! What’s going on? The heat equation run in reverse, creatively called the reverse heat equation, suffers from the butterfly effect. Small uncertainties in the known temperature distribution cause huge variations in the “postdicted” temperature distribution. In the case of the reverse heat equation, this effect is so severe that we can’t make any useful statements.

Let’s try to understand what’s going on.

Understanding the Reverse Heat Equation

Why is the reverse heat equation so chaotic? What causes the butterfly effect here? Let’s think about how heat behaves. Heat spreads out, from hot regions into cooler regions. This makes hot regions cool down and cold regions warm up. Eventually everything becomes uniform.

If you reverse this behaviour, like rewinding a video, heat moves from cold regions to hot regions. Hot regions become even hotter and cold regions become even colder! This means that if you take a surface with a uniform temperature and randomly make some spots just a little hotter than others, those random warm spots will just keep getting warmer. Any difference from the average temperature, no matter how small, gets exaggerated exponentially. This means that if we want to work backwards from a near-uniform temperature distribution to find out how it originally looked, we need to be exactly certain of the temperature everywhere. And we can never be exactly certain. Measurement tools are flawed. And even if we did have perfect tools, quantum mechanics forbids infinitely precise measurements (at least, in finite time).

Worse, since heat diffuses, every original pattern—no matter how strange—leads to a uniform temperature across the metal. So even if the heat spread out perfectly, with every spot exactly the same temperature as every other spot, the reverse heat equation is still useless. Confronted with an infinite number of possible original patterns, it’s forced to just make an arbitrary decision. And while this process isn’t random, the solution that the equation picks will almost certainly be incorrect, since its odds are literally infinity to one.

What Makes Heat Special?

The inability to make postdictions about temperature is surprising. Most of the laws of physics work perfectly well in reverse. If I know the height of waves in a pond—like the one shown in figure 4, for example—at the present moment, then I can say what the pond will be doing at any moment in time, whether past or future. (At least in principle. In reality, friction will convert much of the wave motion into heat. The waves also need to be sufficiently low-energy; otherwise, water can become chaotic. I’ll get to that in a bit.)

So why is heat special? Roughly speaking, the temperature of a metal is actually an average of the energy of the atoms that make it up. In principle, we could track the motion of every individual atom and make a prediction of their motion after heating the metal up with a laser. Then we could make a good postdiction by tracking the atomic motion back in time.

Of course, this is impossible in practice. There’s way too many particles and way too much information to keep track of, so we’d need a practically infinite amount of computing power. So instead, we use the abstraction of temperature, which averages over the particles.

This abstraction has a price, however.  We are intentionally hiding information from ourselves: the precise configuration of the metal. And so it should come as no surprise that we can’t use the heat equation in reverse. We lack the necessary information to do so! We can even quantify how much information we’ve hidden from ourselves. The quantity that tells us this is the entropy of the system. And one way to understand the Second Law of Thermodynamics (“entropy never decreases”) is that, as we step forward in time using the heat equation, we forget more and more about the initial configuration of our metal.

(I want to note that, although I’ve been talking about tracking particles, which are classical, quantum mechanics has analogous ideas. Instead of tracking particles, you track—or average over—a wavefunction whose amplitude represents the probability of measuring all the of the positions of a huge number of particles.)

Manageable Chaos

The reverse heat equation is totally unusable. There is no saving it. But it is an extreme example of the butterfly effect. And it’s not actually chaotic. True chaos is more manageable because it is well-posed, meaning that predictions are, in principle, possible.

Manageable chaos emerges naturally in many areas of science. If the pressure is strong enough, or the temperature or speeds high enough, fluids like air and water are actually chaotic, but in a way that we can handle. Because it takes a lot of computing power to handle the chaos in the atmosphere, it’s very difficult to make concrete predictions about the weather…but it’s not impossible.

Large-scale phenomena, like planetary motion, can also be chaotic. Two objects gravitationally attracted to each other will behave pretty predictably, but adding even one more mass to the system can cause their motion to become chaotic. Satellites under the gravitational influence of both the Earth and the moon, or both the sun and Jupiter, are important examples of such three-body systems.

Understanding chaotic systems is very difficult, but it’s also essential if we are to understand much of the universe. And in many cases, we can manage the chaos.

Related Reading

If you enjoyed this post, you may enjoy some of my other posts on mathematics.

• In this post, I describe the many sizes of infinity.
• In this post, I describe the history of imaginary numbers.

Further Reading

• If you’re curious how I produced those images, I put my code in the IPython notebooks in this bitbucket repository. Feel free to play around with them. I’m afraid there’s no documentation at the moment.
• You can find a more technical discussion of the heat equation and reverse heat equation in this blog post by an engineering Ph.D. student.
• And here‘s an in-depth discussion of entropy as “lost information.”
• And for a much more in-depth discussion of chaos, check out this awesome ebook.